MAINT: AdversarialConversationManager - unify adversarial-chat context, schema, and multimodal feedback#2053
Conversation
Add a shared `adversarial_chat` JSON schema (next_message, rationale, last_response_summary) and wire the Crescendo, TAP, and PAIR adversarial-chat prompts onto it so their prompts are interchangeable. Parsers now read/return next_message and forward the schema to schema-aware targets via prompt metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The message normalizer already appends the response JSON schema when it is forwarded via prompt metadata, so the hand-written schema block in the Crescendo system prompts duplicated that instruction. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Routes Crescendo and TAP adversarial sends through a real PromptNormalizer and a MockPromptTarget (which lacks native JSON_SCHEMA support) to verify the shared adversarial_chat schema is forwarded via prompt metadata and rendered into the prompt the adversarial chat receives. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…eam-attack-schema # Conflicts: # pyrit/datasets/executors/crescendo/crescendo_variant_1.yaml # pyrit/datasets/executors/crescendo/escalation_crisis.yaml # pyrit/datasets/executors/crescendo/therapist.yaml # pyrit/datasets/executors/pair/attacker_system_prompt.yaml # pyrit/datasets/executors/tree_of_attacks/adversarial_system_prompt.yaml # pyrit/datasets/executors/tree_of_attacks/image_generation.yaml
romanlutz
left a comment
There was a problem hiding this comment.
I love this! It solves lots of problems I've noticed but not really had a good handle on (yet!). That said, there are lots of conflicts with #1377. It's compatible if we adjust this or #1377. Some notes below.
-
API migration: #2053 renames
seed_prompt→first_messageand addsadversarial_prompt_template. This would probably need deprecation etc. -
Responsibility boundary: #1377’s
ModalityFeedbackRouterhandles true multimodal forwarding (message pieces). #2053’s manager centralizes adversarial turn/schema logic but is text-message-centric inRedTeamingAttack. We'd need to generalize it to accept/build fullMessageobjects (not just rendered text) so #1377 media forwarding remains intact. -
First-turn placeholder flow: #1377 supports adversarial placeholders + seed media for edit-only starts; #2053’s red-teaming path bypasses that behavior unless explicitly reintroduced.
-
Rollout scope: apply manager only to red teaming first, or also refactor Crescendo/TAP to share the same adversarial-turn primitive.
Another thought that came to mind: Should there be an objective conversation manager then? It's worth considering, but as a follow-up if at all. There’s already ConversationManager; a new objective manager should only centralize objective-target send mechanics (rotation for single-turn targets, converter plumbing, execution context), not absorb attack-specific control flow (backtracking, tree branching, pruning).
So all in all: This PR is compatible with #1377 if integrated deliberately; a mechanical merge would likely regress #1377 multimodal behavior in red teaming. @rlundeen2
Resolve the collision between PR microsoft#2053 (AdversarialConversationManager) and main microsoft#1377 (_ModalityFeedbackRouter) using Option A: the manager owns the adversarial send/parse and, when a modality router is injected, builds its outgoing message via router.build_adversarial_input_message so the adversarial model still sees generated media (preserves microsoft#1377's multimodal feedback loop). red_teaming keeps self._modality_router for the objective side (placeholder fill / build_objective_input_message). Preserve main's blocked/error/use_score_as_feedback semantics by baking them into the default adversarial prompt template (DEFAULT_ADVERSARIAL_ PROMPT_TEMPLATE) rendered through _MessageView, rather than the removed imperative _build_adversarial_prompt_async / _handle_adversarial_* helpers. Note: the default red_teaming adversarial system prompt (text_generation.yaml) declares no JSON schema, so the manager uses the raw-text path (adversarial reply is the next message verbatim); the schema/JSON-parse path only engages when a system prompt or first message declares a schema. Test reconciliation: - red_teaming: drop obsolete handler tests, rewire modality-router integration tests onto the manager/raw-text path, rename seed_prompt -> first_message. - attack_config: add coverage for all six default-template branches (blocked / error / text / text+feedback / no-text+feedback / empty). - simulated_conversation: cover schema vs raw-text next-message handling. - shared attack tests updated for the first_message rename. All 873 attack unit tests pass; ruff + ty clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the blocked/error/empty/score-feedback control flow out of the
per-turn Jinja template into a testable Python helper and render the
template strictly. Detect blocked/errored pieces across all message
pieces instead of only the first, fixing a bug where a blocked/errored
later piece was masked by an earlier clean one.
- adversarial_conversation_manager.py: replace the _MessageView /
_MessageBucket template shim with _build_adversarial_feedback_text
(+ _joined_text_value / _first_response_error); render via
render_template_value (strict) exposing feedback_text + objective.
- attack_config.py: simplify DEFAULT_ADVERSARIAL_PROMPT_TEMPLATE to
"{{ feedback_text }}".
- tests: relocate the template-branch coverage to
TestBuildAdversarialFeedbackText (adding any-piece blocked/error
cases) and update prompt assertions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… lifecycle Address review findings microsoft#4-microsoft#6 on the adversarial conversation manager: - microsoft#6 Schema-driven parsing (single source of truth): `_parse_adversarial_reply` derives its required/permitted keys from the resolved response schema (`required`, `properties`, `additionalProperties`) -- the same schema the manager forwards to constrain the target -- instead of a hard-coded copy of the adversarial_chat keys, so the parser cannot silently drift from adversarial_chat.yaml. Behavior is identical for the shared adversarial_chat schema; `next_message` is always required since the attack loop consumes it (enforced at parse time and, for the property, at construction). - microsoft#5 Naming: rename the manager's `adversarial_first_prompt_template` param/attr/ property to `first_message`, matching `AttackAdversarialConfig.first_message`. - microsoft#4 Lifecycle: build the AdversarialConversationManager once per execution in RedTeamingAttack (`_build_adversarial_manager`, constructed before the turn loop and threaded into `_generate_next_prompt_async`) instead of rebuilding each turn. - Drop the never-read `_adversarial_response_json_schema` field (keep the construction-time "declared on both" validation) and the dead `_DEFAULT_SEED_PROMPT` constant; refresh the stale `adversarial_prompt_template` comment. - Thread the resolved schema through the simulated_conversation parse call. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Crescendo and TAP/PAIR each hand-rolled their own parser for the same adversarial_chat JSON schema. They now delegate to the shared _parse_adversarial_reply used by Red Teaming and Simulated Conversation, so validation and normalization stay consistent across every adversarial-chat executor. This fixes TAP/PAIR's missing camelCase normalization (the exact gap that once broke a Crescendo CI run) and makes them enforce the schema's required keys and additionalProperties instead of only checking next_message. The schema argument is optional, so schemaless custom adversarial prompts keep the permissive next_message-only contract. Also expands coverage: direct TAP node parser tests (camelCase accepted, strict-schema rejection of missing/extra keys, markdown stripping, schemaless-stays-lax), a manager next_message coercion test, a Red Teaming build-once test, and a Simulated Conversation invalid-JSON test. Removes the now-obsolete Crescendo _camel_to_snake test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
rlundeen2's WIP renamed AttackAdversarialConfig.seed_prompt to first_message (field + factory internals) but left four tests in test_attack_technique_factory.py asserting the old field, causing AttributeError/TypeError. Update those tests to the new field name; the factory source was already migrated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…describe them When seed/reference images are provided, the crescendo, red_teaming, and tree_of_attacks image_generation adversarial system prompts now instruct the adversarial model to refer to those images by position and issue an edit/compose instruction, rather than describing their contents. Describing the contents made image-editing targets regenerate from scratch and drop the seeded subject's identity. The seed guidance is conditional, so the no-seed generation path is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Brings in 7 upstream commits (scenario DEFAULT strategy curation, dataset parameters on dataset configuration, initializer fixes, Azure SQL memory test fix, schema migration guard). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Right now this only does RedTeamAttack. Which is good! But are there followups for the other attacks? Or we could do it here. RN Crescendo, TAP, simulated_conversation all use some of it, but should (eventually) use the AdversarialConversationManager to take care of the next turn generally. |
rlundeen2
left a comment
There was a problem hiding this comment.
This is looking good! The comments are scattered so I wanted a single "top-level" comment
AdversarialConversationManager should own the entire adversarial contract. You hand it the AttackAdversarialConfig, and it owns resolution of the schema, the system prompt, the first/next user messages, the modality placeholder weaving, and next-turn generation. Attacks should pre-resolve and set nothing — they just ask for the next message and get back a ready-to-send Message. Concretely:
- Require schemas everywhere. Default to the canonical adversarial_chat schema, managed by the manager, overridable via config — and drop the raw-text path. Motivation: right now an inline string system prompt silently gets no schema and is never validated, and resolve_adversarial_json_schema returns None when neither prompt declares one, so "no schema" happens by accident rather than by choice. This means rewriting red_teaming's default text_generation.yaml to actually emit the JSON (schema + prompt are a matched pair) — I just did the same across all scorers and it was small and low-risk.
- Manager owns setup. The set_system_prompt block and _set_adversarial_chat_first_message should move into the manager (it takes the config and does this). Just mind the two constraints: it must run before prepended-turn hydration, and it renders with max_turns.
- Manager owns _generate_next_prompt_async, including modality placeholders. Attacks shouldn't branch on is_adversarial_placeholder. Folding this in also fixes a latent bug: the red_teaming bypass path returns the message without .duplicate() while Crescendo/TAP duplicate — three copies, already drifted.
- Renames for clarity while they're still free (not released): adversarial_system_prompt / adversarial_first_user_message / adversarial_next_user_message.
- Consider _AdversarialConversationManager — it's an internal component and I expect churn early on.
I'm open to either land it on RedTeaming here, and do Crescendo / TAP / SimulatedConversation as fast-follow, or doing all here.
Your objective-conversation-manager idea is worth a follow-up issue, but I agree it should only centralize objective send mechanics, not attack control flow.
…everywhere The _AdversarialConversationManager now owns the entire adversarial-chat contract: schema resolution (defaulting to the canonical adversarial_chat schema so every reply is validated - no raw-text path), system-prompt setup, first/next user-message rendering, per-turn send/parse/retry, modality placeholder weaving, and the bypass path (now always .duplicate()s the seed, fixing the drifted red_teaming copy). - Rename class to _AdversarialConversationManager; rename params to adversarial_system_prompt / adversarial_first_user_message / adversarial_next_user_message. - Unify get_first_message_async + get_next_message_async into one entry point returning a ready-to-send AdversarialTurn; add override mode (adversarial_prompt_text) for Crescendo/TAP. - Migrate RedTeamingAttack fully onto the manager; rewrite text_generation.yaml to emit JSON + declare response_json_schema_name: adversarial_chat. - Delete AttackAdversarialConfig.get_json_schema(); warn (not raise) when a media-only objective response is dropped to a text-only adversarial. - Reconcile manager/attack_config/red_teaming tests; add regression tests for bypass-duplicate, schema-everywhere fail-fast, empty-feedback warning, override mode. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both attacks now build their bespoke adversarial prompt text and hand it to the shared _AdversarialConversationManager, which owns schema resolution, send/parse/json-retry, the caller-seed bypass (always duplicated with fresh ids), placeholder filling, media forwarding, and objective-message construction. Manager gains two capabilities the attacks need: set_adversarial_system_prompt now accepts **extra_render_values so attacks inject bespoke system-prompt inputs (Crescendo's conversation_context, TAP's desired_prefix/conversation_context) without rendering the prompt themselves; and a new generate_adversarial_reply_async send/parse entry point returns the parsed AdversarialReply without building an objective message, for TAP which scores next_message and may re-prompt before deciding what to send. Removes Crescendo's _get_attack_prompt_async/_send_prompt_to_adversarial_chat_async/_parse_adversarial_response and TAP's _parse_red_teaming_response plus the per-node inline send/parse, eliminating three hand-rolled adversarial parsers so every adversarial reply is validated against the canonical adversarial_chat schema (camelCase-normalized, markdown-stripped, required-keys enforced). Adds regression tests covering the schema-everywhere contract and the new manager entry point. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Crescendo's _send_prompt_to_adversarial_chat_async and TAP's inline send were replaced by the shared manager, which parses every adversarial reply against the adversarial_chat schema. Drive Crescendo's assertion through _build_adversarial_manager(...).generate_adversarial_reply_async and back both attacks with a JSON-returning mock target so the real send/parse cycle completes while still asserting the schema reaches the adversarial target. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Drives RedTeaming, Crescendo, and TAP end-to-end with a schema-aware adversarial reply that emits camelCase keys (nextMessage/lastResponseSummary) and asserts the normalized next_message reaches the objective target. This is the exact reply shape that once broke a Crescendo CI run under a hand-rolled parser; asserting it across every consuming executor guards against any of them regressing away from the shared schema-aware parser. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the is_adversarial_placeholder branch from the TreeOfAttacks node's seed-handling path so the node no longer decides between bypass and placeholder-fill itself. The seed message is handed to the manager's get_next_message_async, which owns that decision, forwards seed media, and fills placeholder slots. Drop the now-dead seed_message threading through the red-teaming prompt helpers and split a merge-artifact test that had two bodies concatenated. Add a bypass (no-placeholder) node test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…anager Route the simulated-conversation final user message through the shared _AdversarialConversationManager so schema resolution, system-prompt setting, the adversarial send, JSON parsing, and pyrit_json_retry all live in one place. - direct_next_message.yaml now declares response_json_schema_name: adversarial_chat and emits the matched JSON Response Format (schema + prompt are a pair), dropping the old raw-text output path. - _generate_next_message_async delegates to the manager instead of calling set_system_prompt / send_prompt_async directly and branching on whether a schema was declared. It takes a prompt_normalizer the manager sends through. - Tests exercise the manager path: JSON reply is parsed, a malformed reply is retried then raises InvalidJsonException (regression guard for the removed raw-text swallow), and a missing response raises a clear ValueError. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The AdversarialConversationManager now owns the full adversarial contract: a new resolve_config classmethod resolves the system prompt, coerces the first/next-message templates (applying canonical defaults in template mode), and fails fast on a duplicate response-schema declaration. Red Teaming (template mode), Crescendo, and TAP (override mode) call it once at construction and store the resolved bundle instead of coercing prompts themselves. Removes the per-attack _set_adversarial_chat_first_message / _set_adversarial_prompt_template / _set_adversarial_chat_system_prompt_template helpers and prunes the now-unused imports, and adds resolve_config unit tests covering both modes, defaults, system-prompt resolution, schema-conflict fail-fast, and invalid-type handling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… TAP RedTeaming already asserts that objective-response media is forwarded to a capable adversarial chat (and dropped for a text-only adversarial chat), but Crescendo and TAP thread last_response media through the same _send_and_parse_async path without asserting it. Add matching TestModalityRouterIntegration cases so all three manager-based attacks cover the adversarial-input direction identically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AdversarialConversationManager
AdversarialConversationManagercentralizes how multi-turn attacks talk to their adversarial chat. Today Red Teaming, Crescendo, TAP, and PAIR each hand-roll the same mechanics: holding the adversarial system prompt, building the per-turn message, sending it on a stable conversation id, and parsing the reply. The manager owns one adversarial conversation and centralizes all of that in one place.Because it's one place, it's also where we decide what context the adversarial chat gets each turn — the objective, the latest score, and the objective target's last response — all via a single
adversarial_prompt_template. The manager computes the per-turn feedback text in Python (handling blocked / errored / empty responses and optional score feedback) and exposes it to the template asfeedback_text, so adding a new signal becomes a template change rather than per-attack code.It also unifies the shared
adversarial_chatJSON schema and is the single home for JSON retry, so schema-aware targets natively constrain the reply and every attack gets consistent parsing/retry behavior for free.Multimodal feedback
The manager folds in the modality router, so adversarial context is no longer text-only:
{{ message.text.converted_value }}).Config surface
AttackAdversarialConfignow carriessystem_prompt(inline string orSeedPrompt),first_message, andadversarial_prompt_template; the legacysystem_prompt_pathis deprecated (removed in 0.17.0). None of this is visible to callers of the executors — the public attack APIs are unchanged.Migrated executors
Every consuming executor now routes its adversarial conversation through the manager: Crescendo, Red Teaming, Tree of Attacks (and PAIR), Simulated Conversation, Context Compliance, and Role Play. The scenario
attack_technique_factoryis updated to match.